Multifidelity Bayesian optimization for hyperparameter tuning of deep reinforcement learning algorithms

E.C. Garrido-Merchán, M. Molina, G. Martínez

Computing and Artificial Intelligence Vol. 3, nº. 2, pp. 2923

Resumen:

This research focuses on comparing standard Bayesian optimization and multifidelity Bayesian optimization in the hyperparameter search to improve the performance of reinforcement learning algorithms in environments such as OpenAI LunarLander and CartPole. The primary goal is to determine whether multifidelity Bayesian optimization provides significant improvements in solution quality compared to standard Bayesian optimization. To address this question, several Python implementations were developed, evaluating the solution quality using the mean of the total rewards obtained as the objective function. Various experiments were conducted for each environment and version using different seeds, ensuring that the results were not merely due to the inherent randomness of reinforcement learning algorithms. The results demonstrate that multifidelity Bayesian optimization outperforms standard Bayesian optimization in several key aspects. In the LunarLander environment, multifidelity optimization achieved better convergence and more stable performance, yielding a higher average reward compared to the standard version. In the CartPole environment, although both methods quickly reached the maximum reward, multifidelity did so with greater consistency and in less time. These findings highlight the ability of multifidelity optimization to optimize hyperparameters more efficiently, using fewer resources and less time while achieving superior performance.

Resumen divulgativo:

Este estudio muestra que la optimización bayesiana multifidelidad mejora el ajuste de hiperparámetros en algoritmos de refuerzo, logrando mejores resultados con menos recursos que la versión estándar.

Palabras Clave: deep reinforcement learning; bayesian optimization; meta learning

Referencia DOI: https://doi.org/10.59400/cai2923

Publicado en papel: 2025.

Publicado on-line: Mayo 2025.

Cita:
E.C. Garrido-Merchán, M. Molina, G. Martínez, "Multifidelity Bayesian optimization for hyperparameter tuning of deep reinforcement learning algorithms", Computing and Artificial Intelligence, Vol. 3, nº. 2, pp. 2923, 2025. [Online: Mayo 2025] doi: 10.59400/cai2923